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■ Introduction 



1.1 The aim of this guide 

This guide has been written to assist projects and 
organisations to understand and implement systems for 
recording soft outcomes and distance travelled. It offers a 
definition of these terms and presents different methods for 
recording soft outcomes and measuring 'distance travelled'. It 
has been written primarily for organisations running projects 
under the ESF Objective 3 programme and particularly those in 
Policy Field 2 (equal opportunities for all and promoting social 
inclusion). In addition, it will be of interest to other projects 
and organisations working with ESF target groups (primarily 
those people who are some distance from labour market 
participation). 

The guide is intended to offer an introduction to soft outcomes 
and distance travelled. It is not designed to be fully 
comprehensive, or to answer all the questions that you may 
have, as that would not be possible within the scope of this 
guide. However, we hope that you will use this guide as a 
basis for discussion within your organisation and importantly, 
you will use it to inform the development of your own soft 
outcome monitoring system. We have presented a range of 
different activities and options but it is important to recognise 
that: 

• It is not possible to touch on all the monitoring systems 
and tools that can be used. Your project may be using a 
system that is not presented here. 

• Not all suggestions will be suitable for your particular 
project. You will have to decide what is most appropriate. 

• Several projects have developed soft outcome systems and 
some of these are listed at the back of this guide. They 
have provided information to us about their systems and 
they are happy to discuss them in more detail with you if 
that would be helpful. We encourage the exchange of good 
practice in this way — you can only learn by doing! 
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1.2 Some definitions 



Throughout this guide, we make reference to various terms 
that may or may not be familiar. As it is important to have a 
common understanding of these, we provide some brief 
definitions below. To begin with there is often a degree of 
confusion as to what is meant by the terms soft outcomes, soft 
indicators and distance travelled. 

1.2.1 Soft outcomes 

These are outcomes from training, support or guidance 
interventions, which unlike hard outcomes, such as 
qualifications and jobs, cannot be measured directly or 
tangibly. Soft outcomes may include achievements relating to: 

• interpersonal skills, for example: social skills and coping 
with authority 

• organisational skills, such as: personal organisation, and 
the ability to order and prioritise 

• analytical skills, such as: the ability to exercise judgement, 
managing time or problem solving, and 

• personal skills, for example: insight, motivation, confidence, 
reliability and health awareness. 

1.2.2 Soft indicators 

There is interplay between indicators and outcomes, in that 
indicators are the means by which we can measure whether 
the outcomes have been achieved. The term soft indicators 
therefore can be used when referring to the achievements 
which may 'indicate' acquisition or progress towards an 
outcome. A project may wish, for example, to explore whether 
an individual's motivation has increased over the length of the 
project. However, this is, to all intents and purposes, a 
subjective judgement, indicators (or measures) such as 
improved levels of attendance, improved time keeping and 
improved communication skills, can suggest strongly that 
motivation has increased. Not all indicators will be suitable for 
all target groups, and some will be target group specific. 

1.2.3 Distance travelled 

The term distance travelled refers to the progress that a 
beneficiary makes towards employability or harder outcomes, 
as a result of the project intervention. The acquisition of 
certain soft outcomes may seem insignificant, but for certain 
individuals the leap forward in achieving these outcomes is 
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immense. A consideration of distance travelled is very 
important in contextualising beneficiaries' achievements. 

It is easy perhaps, to think that: 

Indicators (or measurements) of soft outcomes can be used as tools 
for measuring distance travelled towards labour market 
participation, 

1.2.4 'Models' and "tools' for measuring soft 
outcomes and distance travelled 

Within this guide, we use these terms interchangeably to refer 
to any system that attempts, in some form or another, to 
measure, assess or record soft outcomes and progress towards 
greater employability. These take a myriad of forms, ranging 
from simple paper-based tools, to complex computerised 
systems. 

1.3 The importance of measuring soft outcomes 
and 'distance travelled' 

As a project, you may feel that you have enough monitoring 
responsibilities, and would like to concentrate on the 'job in 
hand', such as providing training or support and guidance. 
However, if you do not already do so, it is important to 
recognise that soft outcomes and distance travelled, and the 
use of systems designed to capture information on them, 
should be an integral part of your project. Monitoring soft 
outcomes and a consideration of the distance travelled by 
beneficiaries should be an intrinsic and mainstream element of 
your project evaluation. 

Table 1.1 below, highlights some of the key benefits of 
incorporating soft outcome and distance travelled monitoring 
into project systems. 

In addition to the benefits that accrue to the project and the 
beneficiary, it is general good practice to measure soft 
outcomes and distance travelled. It improves the process of 
working with beneficiaries and raises the standard of service 
delivery. Consideration of soft outcomes also provides a 
valuable context for clients' needs and progress. It provides a 
truer, more rounded picture of successes. 
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Table 1.1 Benefits of measuring soft outcomes and distance travelled 



General benefits Hard outcomes such as jobs obtained, numbers of qualifications, and numbers 
progressing onto further education and training (though useful in some cases), 
do not show the success of the project as a whole. They are an insufficient 
indicator of a beneficiary's increased employability. Target groups that are 
facing multiple barriers to employment may be a long way from being able to 
acquire a qualification or employment. Consideration of soft outcomes for such 
groups is a crucial indicator of success. Measuring soft outcomes can also help 
with the national level evaluation to provide a fuller picture of the impact of the 
programme as a whole. 



Benefits at a Many projects may have specific objectives that relate to the acquisition of soft 
project level outcomes. It is vital that projects have the systems in place to be able to 
measure progress towards these objectives. 

Soft outcomes can provide a further indicator as to the additionality of the 
intervention, ie positive results that may not have occurred in the absence of the 
activity in question. 



Employers are particularly interested in soft skills and abilities, and have been 
found greatly to value key skills, and personal attributes and attitudes. The 
beneficiary will therefore be at a distinct advantage in the labour market if they 
are able to demonstrate possession of such skills and attributes through 
evidence gained during the project. 

Working with the beneficiary to record and monitor soft outcomes and distance 
travelled, and involving the beneficiary in the assessment process, can be a very 
empowering experience. A project worker may be able to demonstrate to the 
beneficiary that they have pre-existing skills and attributes of which they were 
unaware. In addition, if beneficiaries are made aware of the distance they have 
travelled it can be an enormous confidence boost. 

The development and recognition of soft skills is part of many beneficiaries' long 
term integration into the labour market. 

Source: IES 



Benefits at a 
beneficiary level 
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■ What are Soft Outcomes? 



So, what exactly are soft outcomes? 

Soft outcomes can be grouped in many different ways. Some 
'core' indicators and outcomes are likely to be applicable to 
most target groups and clients. It is perhaps useful to look at 
these in the first instance before considering some of the other 
soft indicators and outcomes that are appropriate for specific 
individuals or project target groups. 

2.1 Core outcomes 

As you will see, there are no set rules regarding which 
indicators relate to particular outcomes, but some of the 
headings or groupings which may be useful in classifying 
'core' soft outcomes are: 

• key work skills 

• attitudinal skills 

• personal skills, and 

• practical skills. 

These outcomes, and some of the indicators that you might 
use to measure them, are highlighted in Table 2.1. Although 
some outcomes, such as key work skills and practical skills, 
are perhaps easier to measure, less tangible attitudinal 
outcomes are equally important as they can move beneficiaries 
towards 'harder', more vocational outcomes and jobs. Personal 
skills and attitudes are as important to employers as their 
vocational counterparts. 

Increased confidence, motivation and self-esteem are 
extremely common indicators that are used in the context of 
ESF projects. Many beneficiaries face multiple barriers to 
labour market participation and suffer high levels of 
disadvantage, which converge to increase the likelihood of 
low confidence and self-esteem. Projects need to work towards 
measuring these outcomes to show the real and full impact of 
their efforts. 
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Table 2.1 Examples of 'core' soft outcomes and indicators 


Types of 'soft' outcomes 


Examples of indicators 


Key work skills 


The acquisition of key skills eg team working, problem solving, 




numeracy skills, information technology 

/ / Z) I 




Numbers of work placements 




The acquisition of language and communication skills 




Completion of work placements 




Lower rates of sickness related absence 


Attitudinal skills 


Increased levels of motivation 




Increased levels of confidence 




Recognition of prior skills 




Increased feelings of responsibility 




Increased levels of self-esteem 




Higher personal and career aspirations 


Personal skills 


Improved personal appearance/presentability 




Improved levels of attendance 




Improved timekeeping 




Improved personal hygiene 




Greater levels of self-awareness 




Better health and fitness 




Greater levels of concentration and/or engagement 


Practical skills 


Ability to complete forms 




Ability to write a CV 




Improved ability to manage money 




Improved awareness of rights and responsibilities 



Source: IES 



2.2 Target group-specific outcomes 

The outcomes and indicators presented here are by no means 
an exclusive listing. Some indicators will be more suited to 
some target groups than others, and Table 2.2 below 
highlights this. It is important to stress again that as projects 
are working with people, and everyone is different, the 
suggested indicators may not be appropriate for every 
individual. Equally, some individuals will have additional 
barriers to overcome and therefore other indicators of 
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Table 2.2 Target group-specific outcomes 



Target group 



Possible indicators 



Disaffected young people 



Improved behaviour 

Improved attendance at school or at project sessions 
Improved engagement with learning or project activity 



Women returners 



Recognition of prior skills and experience 



Drug/alcohol (ex)abusers 



Better health 

Lower levels of drug or alcohol use 
Improved personal appearance/presentability 



Ex-offenders 



Lower rates of reconviction 
Lower rates of reoffending 
A slowing of reoffending 



Homeless people 



Permanent accommodation 



People with learning disabilities 
or mental illness 



A greater level of self-awareness 
Reduced anxiety and depression 



Source: IES 



progression and soft outcomes will be necessary. In essence, it 
is important to recognise the uniqueness of the individual and 
the great variety within 'target groups'. 



2.3 Other issues to consider 

The decision as to which outcomes should be considered, and 
how they should be measured, does not rest on the client 
group alone. Project objectives will need to be taken into 
account too. It may be useful to ask the following questions: 



• Is the achievement of soft outcomes explicit in the project 
objectives, and if not, should it be? 

• What is the ultimate aim of the intervention or project, and 
what soft outcomes will be relevant? 



Clarity around the project objectives will allow a rigorous and 
meaningful framework to be established around which 
monitoring and evaluation can take place. 
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■ How to Measure Soft Outcomes 



3.1 Process or method 



Methods to measure soft outcomes and distance travelled are 
numerous and varied and we must reiterate that one system 
will not suit all. What may work well for one project may not 
work for another. The system that you choose, and how you 
implement, will depend very much on the activities and 
objectives of your project, your target group, the length of the 
project, and the resources you have at your disposal. The 
options we provide here represent some of the methods you 
may wish to consider when designing a soft outcome 
monitoring system for your project. 

3.1.1 Establishing a baseline 

This is a vital process if soft outcomes and distance travelled 
are going to be measured. It is crucial to establish a baseline of 
soft skills, aptitudes and attitudes from which individual 
progress can be measured. This can normally be done during 
the initial assessment phase when clients' needs are 
established, barriers to employability identified and personal 
development targets are set. An examination of 'soft' skill 
needs falls naturally from this phase of the project. 

3.1.2 How to collect and record information on 
soft outcomes and distance travelled 

It is vitally important that your methods for collecting and 
recording information on soft outcomes and distance travelled 
are rigorous and targeted to your client group. Some of the 
methods, which can be used to collect information on soft 
outcomes, are shown in Table 3.1 below. 

Selecting and implementing a range of methods to collect data 
on soft outcomes is more likely to capture a full picture of 
clients' progress. 
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Table 3.1 Collection methods 



Type of collection 
method 



Comments and issues to consider 



Individual action planning, 
personal action planning 
and goal setting 

Reviews between 
trainers/assessors and 
clients to record soft 
outcomes 

Daily diary or personal 
journal 

In-depth reflection during 
or after the course 



Recorded observations of 
group or individual 
activities 



Presentation of material in 
a portfolio 



Tests 



The drawing up of individual action plans is normally carried out during 
the initial assessment session and then reviewed at regular intervals to 
gauge whether goals have been met. An action plan can include 
personal objectives, priorities and reflections on progress. 

Improvements over time can be noted and recorded during regular 
formal or informal reviews. This system is largely reliant on a sound 
judgement from the client or project worker and will not provide an 
absolute or formal measure of distance travelled. 

Clients can be encouraged to write about progress towards soft 
outcomes. Issues of confidentiality should be considered. 

Beneficiaries could be asked to consider and review their progress as 
they come to the end of their training course, or a particular element of 
the project (such as a work placement). This could be incorporated as 
an assignment that could be included in a beneficiary's portfolio of 
evidence of achievement. Baseline information is particularly useful here 
as data can be compared over time. Questionnaires are an important 
tool for this purpose. 

It is important to have comprehensive documentation systems that will 
allow for the recording of anecdotal evidence of outcomes achieved and 
progress made. 

This method requires a high level of observer skill, and there is the 
danger of observer bias, and also that the observer will influence the 
behaviour being observed. If the beneficiary/ies are unaware that they 
are being observed, this may negate the problem. 

This could include evidence of tasks completed successfully indicating 
achievement of outcomes, or progress towards them. An evidence- 
based portfolio would be a concrete output that could be presented to 
an employer. 

Some projects use psychometric testing within the assessment process. 
This is generally a diagnostic procedure but could be adapted to 
establish a baseline and measure distance travelled. Tests may be 
useful in establishing a beneficiary's existing skill level. The test could 
then be conducted at a later stage to illustrate any progress made. 



Source: IES 



3.1.3 The assessment process 

Assessment is usually a two-way process between trainer and 
client to ensure that judgements are reliable, robust and 
professional. Assessment should serve as a support function 
for the client to assist them in identifying real needs and 
bringing them to the attention of project workers. Beneficiaries 
must be willing participants and be able to see the value in the 
process. 

Any system of assessment needs to be built around positive 
affirmation of beneficiaries' current skills and attributes and 
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highlight examples of positive progression. This creates an 
atmosphere where people feel valued and motivated. 
Essentially, assessment and review systems should enable 
beneficiaries to recognise the skills they already possess and 
identify their own capacity for learning and progress. 
Assessment systems should also be mindful to guard against 
highlighting unnecessary and negative regression. This is not 
to say that regression should always be ignored, as clearly it 
may sometimes be due to circumstances beyond the clients' 
control. It is important therefore to identify the cause of the 
regression and then work with the client to support positive 
progression once more. 

Time is also a consideration when devising a soft indicator 
system, as some techniques require a fairly intensive input of 
time. Quite clearly, this would not be suitable for a project 
where contact is only sporadic such as those associated with 
outreach working. 

3.1.4 Different assessment formats 

Most systems focus on paper-based methods of assessment, 
where beneficiaries complete written questionnaires, or write 
narrative accounts of their progress or their skills. Although 
this is appropriate for some target groups, there are 
circumstances in which the use of other assessment formats 
may be more suitable. Beneficiaries with numeracy, literacy or 
learning difficulties may experience problems in undertaking 
written assessments. One solution may be to work in 
conjunction with clients, using other media which is perhaps 
more appropriate to their needs. 

In addition to paper-based assessment tools, you may wish to 
consider the examples shown in Table 3.2 below. 

3.1.5 Measuring soft outcomes and distance 
travelled 

Unlike 'harder' job and qualification outcomes, it is difficult to 
achieve an 'absolute' measure of achievement and progress in 
the realm of soft outcomes. In many cases, projects can only 
record that an improvement has occurred rather than being 
able to quantify how much improvement. 

However, some projects attempt to measure soft outcomes and 
distance travelled more systematically (and numerically) 
through scoring systems and scales. Commonly, beneficiaries 
are asked to complete questionnaires based on scales of feeling 
or agreement. A statement can be presented, such as, T feel 
optimistic about my future' and the beneficiary can then 
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Table 3.2 Assessment tools 



Assessment 
format 



Example 



Comments 



Computer-based 
systems 



Games 



3D media 



eg where a beneficiary 
may complete 
assessment forms 
on-line 



eg self-assessment 
cards 



eg where a beneficiary 
may move an 'actual' 
sliding scale along a 
board 



PC based systems of self-assessment have the added 
advantage of giving clients the opportunity to learn and 
develop IT skills at the same time (although projects 
should not assume that clients possess IT skills). 

This medium may allow for a more systematic 
aggregation of data. 

This method may not be as intimidating as a written 
assessment. If carried out together with a skilled project 
worker, the use of games may encourage clients to talk 
about themselves even when they are lacking self- 
awareness. 

Again, this assessment format may be more suitable for 
those with numeracy and literacy difficulties, and may be 
less intimidating than a paper based assessment tool. 



Source: IES 



indicate their response on a scale of 1 and 10, for example, or 'I 
agree strongly' through to 'I disagree strongly'. The client's 
starting point can then be established (in the case of initial 
assessment) or a measurement of distance travelled over time 
be calculated (if used to review progress and ultimate 
outcomes). 

It is important to ask the right questions and to avoid being 
ambiguous, meaningless or patronising when devising 
questionnaires and soft indicator systems, a point to which we 
return when we discuss 'tools'. 



3.1.6 Inferences 

Needless to say, the achievement of soft outcomes and 
progress towards employability cannot always be (entirely) 
attributable to the project. Other external factors can exert 
strong influences on beneficiaries' attitudes and achievements. 
A client may be more confident because of positive changes in 
his personal life rather than because of the training and 
support they are receiving from the project. It is important to 
understand a client's external situation in order to gain a 
better understanding of the context in which progress is being 
made and the factors that are most likely to be attributable to 
the project. 
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Table 3.3 Type of assessment 



Type of assessment Comments and issues to consider 



Beneficiaries may answer in a way they feel is expected of them, and answers 
to questions may not be a true refection of their ability: just how they were 
feeling at the time. It is possible that a more honest and rounded response 
would be elicited if there were an honest and trusting relationship between 
the beneficiary and the member of staff responsible for the assessment. 

Self-evaluation is a skill in itself, and not all beneficiaries will have sufficient 
levels of self-awareness to be able to self-assess in a meaningful way. 

Employer assessment This could provide a very useful addition to the soft outcome and distance 
travelled picture. If beneficiaries are undertaking work placements, employers 
could be asked to comment on the progress that beneficiaries have made. 

Staff need to be well trained in the system which is being used and all need 
to working to common definitions. Due to the danger of subjectivity in 
assessment, external quality control or verification should be put in place to 
ensure a consistent approach. It is important to note that it is not always 
possible to remove bias from the way information is collected. 

Staff need to be fully committed to the process and concept of soft outcomes 
and distance travelled. 

Peer assessment Positive reinforcement from peer group members may be particularly 
encouraging. However, as a measurement of progress this method of 
assessment should not be used on its own, as there is a danger of 
subjectivity and bias. 



Client or beneficiary 
self-assessment 



Project staff 
assessment 



Source: IES 



3.2 Who measures? 



Who actually carries out an assessment of soft outcomes and 
distance travelled will depend on your particular project. 
Beneficiaries are clearly the mainstay of self-assessment, 
usually in conjunction with, and perhaps supplemented by, 
assessments from project staff and external individuals. Those 
involved in assessment should be those who have regular 
contact with the beneficiary and whose professional 
judgements may add to the stock of knowledge about their 
progression. We have highlighted in Table 3.3 above, some of 
the issues that may emerge with different assessors. 



3.3 When to measure? 

In order to measure progress and distance travelled, it is 
essential that assessment takes place at the very least at the 
beginning and end of your project. If time and resources 
permit, it is advisable to carry out assessment at regular 
intervals during the project. Not only does this allow for a 
more responsive service to changing needs, it also provides a 
more thorough and reliable evaluation of whether true 
progress is being made. 
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3.4 Management of systems 



The shift to consideration of soft outcomes and outputs may 
require a cultural shift both internally and externally, and 
result in a significant learning curve for project staff. If 
monitoring and evaluation is carried out externally, these 
personnel must be signed up to the concepts of soft outcomes 
and distance travelled. 

Communication is particularly vital if systems are to operate 
effectively. Good communication is extremely important 
between beneficiaries and staff, between staff, and between 
projects and other organisations to share good practice. Clear 
instructions and guidance must be given to project staff to 
enable them to carry out assessments of soft outcomes and 
distance travelled. 

3.5 Presentation and use of soft outcome and distance 
travelled information 

It is important that you consider how the information that has 
been collected will be used and how it will be presented. You 
may want to use the information to: 



• Provide evidence that the aims and objectives of the project 
have been fulfilled. 

• Assess the added value of the project at an aggregate level. 

• Feed into the overall project evaluation and dissemination 
strategies, and inform appropriate audiences of the project 
lessons. 

• Provide feedback to individual beneficiaries to increase their 
confidence and satisfaction with their own achievements. 
Importantly, through feedback they will be able to see how 
they are meeting their individual aims and objectives, and 
fulfilling their personal goals. 



How the information will be used will help determine its 
presentation format. You may want to present it as: 



• a written report which collates evidence at the project level. 
This could be included in an evaluation report, or could be 
presented as a stand alone document 

• case studies of individual beneficiaries within a broader report 
to highlight success 

• portfolios where beneficiaries can use the evidence of progress 
and outcomes in interviews or to construct CVs. 
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■ The Tools Themselves 



There are two basic requirements for the tools that you devise 
to measure soft outcomes and distance travelled. These are: 

• reliability, and 

• validity. 

Your tool should obtain consistently robust results over time, 
with different client groups and with different staff members 
applying it, and the questions you ask must measure the 
outcomes, or performance dimensions, for which they were 
intended. The tool you use should allow you to improve 
your learning and understanding of the client group that you 
are working with, which will then allow you to provide 
more tailored provision for them. 

This need not be a complicated process. Clarity of thought and 
simplicity of design can go a long way towards ensuring a 
high quality soft indicator system. This technical note provides 
some overarching principles to help you design the tools for 
this task. 



There are four tasks that you need to undertake: 

• Think about the outcomes, or dimensions of performance, 
that you wish to measure. 

• Translate these into meaningful and precise questions. 

• Review the overall mix of questions to ensure they work 
together. 

• Decide on the order in which questions will be asked. 
The questions you use should pass the following tests: 



4.1 Phrasing the questions 
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1: Is this the most direct way of asking the question? 



Think very carefully about the area in which you are 
interested. Does the question you have written ask about this? 
You should avoid proxies, which are simplified or trivial 
questions that stand in for more important subjects. 

2: Is this the most concrete way of asking the question? 

Without straying from the area in which you are interested, 
you should ask your question in the most concrete way 
possible. The more factual and descriptive the question is, the 
less subjective the answer will be. 



An example 

You might not obtain a clear response if you ask 'are you on time?' 
Punctuality could be considered to be composed of four elements: 

• how often the person is late 

• how late they are 

• whether they are late due to circumstances outside their 
control 

• whether they telephone to explain the reason. 

Perhaps a better way to ask a question regarding timekeeping 
would be to use a number of separate statements with which the 
client agrees or disagrees: 

• I always arrive for the beginning of work/the course. 

• I always telephone with a reason if I know I am going to be 
late. 

• I only miss work/the course due to events outside my control. 

• I am never late. 

• My personal circumstances mean I often have to leave 
work/the course early. 



3: Is the question really an element of employability? 

Can you show a logical link between the dimension you are 
investigating and employability? It is arguable that we should 
not include personality dimensions in soft indicator systems 
because this implies that there is only one acceptable form of 
personality, which is not true: pessimists do get jobs! 
Measuring behaviour and attitude has greater validity because 
these can be changed and are less intimate. 
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4: Is the question measuring something that will change 
as a result of your work? 

Your soft indicator system will need to measure progress 
across a wide range of clients and individual circumstances. 
Without making a link between what you do as part of the 
project and the possible effect this has on clients, the soft 
indicator system will not show you at your best. If you do not 
have a good picture of the soft skills affected by your work, 
the soft indicator system should be able to help you, so review 
the initial results with this in mind. 

5: Are you measuring each element of your project 
work? 

Look at the overall pattern of questions. Does it give a 
balanced picture of the work that you do? If the system 
concentrates on one element at the expense of others, you 
might inadvertently be valuing some parts of the project and 
particular staff more than others. This could distort the work 
of your project by encouraging staff to divert effort into the 
elements being measured. 

6: Is the question taking too much for granted? 

The questions should not assume that the respondent has a 
particular possession, attitude or set of behaviours. For 
instance, questions for homeless people need to be reviewed 
very carefully to ensure that they are not implying that the 
respondent has a home, car, established set of friends, or daily 
routine. Questions about drink problems should not assume 
that all respondents drink alcohol: a filter question might be 
needed to ask if respondents drink alcohol (telling them to 
skip the next question(s) if they do not). 

7: Is a positive answer necessarily good? 

Equivocal questions are difficult to interpret. For example, 
asking the respondent if they have 'stopped themselves when 
they wanted to shout at someone' or 'been pleased with 
something they have done' does not tell you whether these 
were appropriate responses. They could reflect passivity or 
poor self-awareness rather than assertiveness. 
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An example 

One system had the following questions for clients to agree or 
disagree with: 

• This week I filled in a form. 

• I did something fun. 

• I have been pleased with something I have done. 

• I have borrowed money. 

• I have found it difficult to concentrate on things. 

Each of these is equivocal. The form could have been in a police 
station; the something fun could have been drugs; the self- 
congratulations could have been misguided, showing poor self- 
awareness; the money borrowed could have been to buy clothes 
for a job interview, a positive action; the difficulty concentrating 
could have been good if it meant they were trying to listen to 
lessons where previously they would not have bothered. 



8: Does the question duplicate information asked 
elsewhere in the tool, or elsewhere in the client records? 

A lean system is more cost effective. Repetition can be 
annoying both to the client and to project workers. Sometimes 
soft indicator systems (and other systems such as 
psychometric tests) have several questions that address similar 
issues, so as to check internal consistency; but this is not 
essential and should not be over-done. 

9: Is the question unambiguous? 

Only ask one question at a time. Never use questions that have 
two or more parts to them. For instance, the question: 'Have 
you been angry or violent with others?' is really two 
questions, one asking about anger, and another asking about 
violence. 

10: Is the question in plain English? 

Phrasing your questions simply will reduce the time taken for 
the client to read them, and avoid misunderstanding or 
misreading. Avoid jargon, slang, or idiom as these will not be 
understood equally or consistently by all respondents. It is 
useful to check that all staff are working with the same 
definitions of any ambiguous words used in questions, and 
that the client has understood what is meant by the question. 
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11: Is the question without any possible offence to 
women, ethnic minorities or other groups? 

As well as the obvious mistakes like sexist language, avoid the 
more subtle offences of taking cultural norms as standard. For 
instance, what some cultures consider as confidence and 
assertion might in others be considered unacceptable bragging 
and pride. 

12: Is the question worded from the client's perspective? 

You will get a better response from clients if the questions 
imply some benefit to them, rather than simply a requirement 
for them to fit into society. Thus questions about 'taking 
responsibility for yourself could be seen as patronising or 
preaching. The same issues phrased in terms of 'overcoming 
setbacks' could be seen as helping them develop streetwise 
behaviour. Similarly, wording questions in terms of clients' 
'realism' could be offensive as it implies there is only one 
correct way of seeing things. 

A second element of the client's perspective is that the order of 
questioning must make sense. It is a good idea to put simple, 
less personal questions at the beginning of the questionnaire 
and more personal ones at the end. 

4.2 Scoring the questions 

Some form of measurement (quantification) is needed in soft 
indicator systems if a meaningful assessment of distance 
travelled is to be gained. The three reasons for this are: 

• to avoid the subjectivity of words 

• to give precision to the descriptions 

• to allow comparisons between users, and across time 
periods. 

Having said this, measurement does not need to be heavy 
handed. Matters of degree can be shown through: 

• scales: for example, rating from one to five, or one to seven 

• symbols or pictures: for example, smiley faces 

• physical artefacts, such as sliding counters on a board 

• the accumulation of evidence. 

The following issues should be considered when designing 
your scales: 
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1: Is the context clear? 

People behave differently in different environments. If the 
context is not clear, the respondent might be frustrated with 
the question and tempted to reply 'it depends'. And 
remember: you are only really interested in the user's 
behaviour at work, not what they do during their leisure time. 



An example 

A question about drugs might be better received and easier to 
apply if tied specifically to the work context, for example: 

• My drug-taking affects my co-ordination and judgement 

• When I use drugs I have difficulty getting up in the morning 

are better questions than the more general: 

• My drug use is out of control. 



2: Is the time period clear? 

Questions about behaviour, attitudes, or feelings should be 
tied to a specific time period. This should be long enough to 
allow the emotion or response to appear, but not so long that 
the respondent will be unable to remember what they were 
doing. 

3: Are the scales anchored? 

People naturally use scales differently: some people group their 
responses around the middle of the scale whilst others stick to 
the extremes. You can encourage respondents to use the full 
range of the scale by giving a brief description for each number. 



For example 

A project that is interested in gauging the extent of drink or drug 
use might signpost scales in the following way: 

1. No problem of this kind during the period rated. 

2. Some over-indulgence within the social norm. 

3. Loss of control over drinking or drug-taking but not seriously 
addicted. 

4. Marked craving or dependence on alcohol or drugs, with 
frequent loss of control and risk taking under the influence. 

5. Incapacitated by alcohol or drug problem. 
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4: Do the scores need weighting? 

You can make some dimensions or questions more important 
than others by multiplying their score up before you calculate 
the total score. 

5: Is an aggregate score meaningful? 

Aggregating scores across all the questions can hide more than 
they reveal, and some soft outcome areas might go down as 
others go up. An example of this is where greater awareness 
leads to a client acknowledging a problem that was previously 
denied. Aggregate figures should therefore be used with care. 
Project workers should always be given the chance to 
comment on figures so that apparently negative results can be 
explained. 

6: Is the scoring positive? 

It is more meaningful to clients if high numbers used for 
scoring equate with high performance. You should also think 
about balancing the scoring so that even poor performance 
receives a result rather than nil. The soft indicator system is 
intended to motivate clients not to demoralise them! 

7: Consider using a norm or standard 

People do not need to be perfect to get a job. They only need to 
be good enough and to meet some form of standard or 
competency. In some cases, the standard is implied in the 
anchoring used. For instance, in the question regarding drink 
and drug use given above, a score of 1 or 2 might be 
considered acceptable, depending on the type of job for which 
the candidate is applying. If the norm or standard is not 
obvious, then you might like to apply the soft indicator 
questionnaire to a sample of your colleagues or friends to give 
a baseline of people in employment. 

8: Is the scoring too obvious to the respondent? 

Phrase some of the questions positively and some negatively. 
You do not want respondents to be able to guess that there is a 
'right answer', obtained by ticking the right hand column of 
the scales all the way down the page. 

9: Minimise arithmetic 

The value of soft indicator systems can be reduced by 
mathematical errors. Project staff are busy people and might 
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not appreciate the task of adding up scores. Computerised 
systems that aggregate scores and reduce errors are often 
preferable. 

4.3 Final word 

As you will appreciate, developing and using a soft indicator 
system will take a good deal of time from you, your staff and 
your users, so it is important that you get it right. You may 
want to look at some examples of other systems to assist with 
the development of your own, and we have given some 
contact details overleaf to help you with this. Exchanging 
examples and good practice in this way should minimise 
duplication and any potential reinvention of the wheel. 
However, you must be prepared to trial early versions of the 
system and obtain feedback as you go along. Ultimately 
though, measuring soft outcomes in a systematic way, in 
addition to gathering data on the harder job and qualification 
outcomes, will allow you to identify the real progress and 
achievements of your project and your clients. 
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Points of Reference 



These projects have helped with the development of this 
Guide. If you would like any further information on the 
systems that they have devised to measure soft outcomes and 
distance travelled, please write to them, or e-mail directly. 

Stewart Bowman 

Mentoring Action and Stepping Stones 

Tyneside Careers 
Careers Centre 
Interchange Centre 
West Street 
Gateshead 
NE8 1BH 

email: s.bowman@tyneside careers.co.uk 

Keith Stead 
The Rickter Scale 

Choiceworks 

75 Townsend Crescent 

Kirkhill 

Morpeth 

Northumberland 

NE61 2EX 

email: keith@kstead.freeserve.co.uk 

Amanda Pavon-Lopez 
Barrier Busters 

NOW 

Addington High School 
Fairchilds Avenue 
New Addington 
Surrey 
CRO OAH 

email: CETS@dial.pipex.com 
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Tom Farrier 

HARP (Care Skills for Working Life) 

Bishop Auckland College 
Woodhouse Lane 
Bishop Auckland 
Durham 
DL14 6JZ 

email: driderwentside@connectfree.co.uk 

Fiona Philipson and Fiona Langskaill 
Bridges Project 

55 Albany Street 
Edinburgh 
EH1 3QY 

email: bridgesproject@compuserve.com 

Chris Llewellyn 
Learning Partnership West 

Floor 1 

4 Colston Avenue 

Bristol 

BS1 4ST 



Linda Keith and Karen Burgin 
Health and Social Gains 

Rehab UK 
Melbourne House 
Melbourne Street 
Newcastle-Upon-Tyne 
NE1 2JQ 

email: lindakeith@rehabuk.org 
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